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FOREWORD 


The  Manpower  &  Educational  Systems  Technical  Area  of  the  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  performs 
research  and  development  in  areas  that  include  educational  technology 
and  training  simulation  with  applicability  to  military  training.  Of 
special  interest  is  research  in  the  area  of  computer-based  systems  for 
maintenance  training.  The  development  and  implementation  of  such 
systems  is  seen  as  a  means  of  reducing  training  time  and  costs  by 
providing  more  highly  individualized  training  than  would  be  otherwise 
possible,  while  at  the  same  time  reducing  the  need  for  operational 
equipment  for  training. 

\ 

The  report  provides  an  annotated  bibliography  of  research  directly 
related  to  human  problem-solving  behavior  in  fault  diagnosis  (trouble¬ 
shooting)  tasks.  It  provides  an  information  source  to  assist  planners 
of  future  research  efforts  in  this  area. 

This  research  is  responsive  to  the  requirements  of  RDT&E  Project 
2Q161102B74F,  "Basic  Research  in  the  Behavioral  and  Social  Sciences." 


JOSEPH (^EJDNER 
technical  Director 
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BRIEF 


Requirement : 

In  conjunction  with  an  ongoing  program  of  research  on  the  use  of 
computer  simulations  for  fault  diagnosis  training,  a  thorough  review  of 
the  existing  literature  was  necessary  to  provide  historical  input  and  to 
identify  current  trends  in  fault  diagnosis  work.  The  results  of  this 
search  are  presented  here  as  a  selective  annotated  bibliography. 


Procedure : 

The  search  procedure  consisted  of  on-line  computer  search  of  the 
past  10  years  of  Psychological  Abstracts  and  ERIC  Documents  through 
mid-1978.  Manual  searches  used  references  from  reports  as  well  as  from 
the  card  catalog  of  the  University  of  Illinois  library.  In  addition, 
experts  in  the  area  of  fault  diagnosis  were  sent  the  references  resulting 
from  the  computer  and  manual  searches  and  asked  to  suggest  any  additional 
references  that  should  be  included. 


Findings: 

The  references  revealed  by  the  search  procedure  were  predominately 
from  the  areas  of  engineering,  psychology,  and  education.  Three  major 
categories  of  materials  are  presented.  These  consist  of  (1)  research 
reports,  (2)  manuals  for  fault  diagnosis,  and  (3)  material  indirectly 
related  to  fault  diagnosis,  such  as  medical  diagnosis  and  mathematical 
problem  solving,  or  items  directly  related  to  fault  diagnosis  that  are 
not  research  reports . 


Utilization  of  Findings: 

This  annotated  bibliography  will  serve  as  an  information  source 
i  for  planners  of  future  R&D  efforts  in  this  area. 
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INTRODUCTION 

The  authors  and  several  colleagues  at  the  University  of 
Illinois  are  investigating  the  use  of  computer  simulations  for 
fault  diagnosis  training.  This  work  runs  the  gamut  from 
context-free  simulations  and  context-specific  simulations  into 
the  live  environment  of  real  equipment  troubleshooting.  This 
research,  supported  by  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  includes  the  development  of 
models  of  human  fault  diagnosis  performance  and  ultimately  an 
intelligent  interactive  training  simulator.  To  this  end,  we  have 
already  run  six  experiments  involving  almost  160  subjects  who 
solved  approximately  13,000  fault  diagnosis  problems. 

As  part  of  our  work,  we  have  prepared  this  annotated 
selected  bibliography  on  human  performance  in  fault  diagnosis 
tasks.  The  sources  in  this  bibliography  are  predominantly  from 
the  disciplines  of  engineering,  psychology,  and  education. 
Although  the  primary  emphasis  is  on  the  research  literature,  we 
have  included  a  representative  number  of  actual  troubleshooting 
manuals  from  the  automotive  and  aviation  fields. 

The  literature  search  included  several  phases,  involving 
both  on-line  and  manual  searches.  Computer  searches  of  the  past 
ten  years  of  Psychological  Abstracts  and  ERIC  documents  through 
mid-1978  were  performed.  Manual  searches  were  conducted  using 
bibliographies  from  various  reports.  Using  the  card  catalog  at 
the  University  of  Illinois  library,  we  obtained  materials  dating 
from  the  early  fifties. 
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The  third  phase  of  this  literature  search  involved  the 
solicitation  of  additional  references  from  noted  fault  diagnosis 
and/or  computer  simulation  experts.  Over  twenty  such  individuals 
were  sent  the  list  of  references  resulting  from  the  first  two 
phases  of  our  literature  search.  They  were  asked  to  skim  the 
list  of  references  and  suggest  any  additional  references  that 
they  felt  to  be  essential  for  a  selected  bibliography  on  human 
performance  in  fault  diagnosis  tasks.  Approximately  one-third  of 
those  queried  responded.  We  greatly  appreciate  their 
contributions . 

The  materials  in  this  bibliography  are  divided  into  the 
following  three  major  areas: 

1 .  Research  Reports 

A.  Primary  Research 

B.  Secondary  Research 

2.  Manuals  for  Fault  Diagnosis 

3.  Materials  Indirectly  Related  to  Fault  Diagnosis 

Primary  research  reports  include  empirical  and/or  theoretical 
studies  aimed  directly  at  human  problem  solving  behavior  in  fault 
diagnosis  tasks.  Secondary  research  reports  refer  to  literature 
reviews.  As  noted  earlier,  a  representative  set  of  training  and 
field  manuals  is  included  in  the  "manuals”  category.  Indirectly 
related  materials  refer  to  studies  of  human  behavior  in  related 
domains  (e.g.  medical  diagnosis  or  problem  solving  in 
mathematics)  reports  that  directly  relate  to  fault  diagnosis 
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but  do  not  provide  research  results  of  a  type  necessary  for  the 
report  to  be  categorized  as  a  "research  report."  Since  the  number 
of  items  that  could  be  placed  in  the  "indirectly  related" 
category  was  potentially  enormous,  we  limited  consideration  to 
items  that  related  to  human  behavior  and  particularly  matched  our 
interests.  Thus,  for  example,  the  large  body  of  literature  on 
maintainability  and  automated  fault  diagnosis  has  been  omitted 
from  this  bibliography. 

Where  possible,  we  used  the  abstract  that  accompanied  an 
item.  However,  we  usually  had  to  supplement  the  abstract 
considerably  in  order  to  make  it  self-explanatory.  The  indexing 
of  each  item  was  based  on  a  consensus  of  the  authors. 


We  hope  to  periodically  update  this  bibliography  and 
re-issue  it.  To  that  end,  we  would  appreciate  suggestions  and 
contributions.  If  possible,  when  submitting  a  suggestion  for 
inclusion  of  a  new  item,  please  include  a  copy  of  the  item. 
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PRIMARY  RESEARCH 


Bond,  N.A.,  Jr.;  Rigney,  J.W.  (1966)  "Bayesian  Aspects  of 

Troubleshooting  Behavior."  Human  Factors .  8:377-383. 

Thirty-nine  Navy  technician  trainees  filled  out  a 
symptom-malfunction  matrix  on  a  blocking  oscillator 
circuit.  The  particular  sequence  of  checks  used  by  each 
man  on  each  problem  was  combined  with  his 
symptom-malfunction  matrix,  via  a  Bayesian  algorithm,  to 
yield  computer  estimates  of  failure  likelihoods  for  each 
component.  The  computer  program  predicted  actual 
parts-replacement  behaviors  in  about  half  of  the  cases. 
Those  technicians  who  start  out  with  valid 
symptom-malfunction  matrices  are  more  likely  to  resemble 
the  Bayesian  processor.  Subject's  troubleshooting 
behavior  that  was  characterized  as  Bayesian  correlated 
with  higher  class  standing  and  longer  time  between  tests. 

It  is  suggested  that  symptom-malfunction  matrix  accuracy 
could  be  a  training  criterion.  Students  could  be  trained 
until  they  could  produce  a  symptom-malfunction  matrix  to 
a  high  standard  of  physical  validity. 

7  pages,  3  references 


Brooke,  J.B.;  Duncan,  K.D.;  Marshall,  E.C.  (1978)  "Interactive 
Instruction  in  Solving  Fault  Finding  Problems."  International 
Journal  Man-Machine  Studies  .  10:603-61  1. 

Thirty  subjects  were  divided  into  two  groups  which  solved 
60  trials  of  fault  diagnosis  problems.  One  group  was 
given  process  information  provided  by  the  interactive 
software  system  when  an  incorrect  diagnosis  was  made.  At 
that  point  during  the  interaction,  feedback  regarding  the 
possible  failures  was  given  to  the  subjects.  Also,  if 
subjects  continued  to  ask  for  a*dditional  information  when 
the  solution  was  obvious,  the  system  informed  subjects  of 
the  fact  that  the  failure  could  be  identified.  The 
control  group  had  no  feedback  and  were  not  informed  about 
the  correctness  of  their  diagnosis.  Analysis  of  variance 
showed  a  significant  effect  of  training  on  learning. 


9  pages,  2  references 
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Brown,  J.S.;  Burton,  R.R.;  Bell,  A.G.  (1975)  ’’SOPHIE:  A  Step 
Toward  Creating  a  Reactive  Learning  Environment . ”  International 
Journal  of  Man-Machine  Studies .  7:675-696. 

This  paper  describes  a  fully  operational  AI-CAI  system 
which  incorporates  artificial,  intelligence  techniques  to 
perform  question  answering,  hypothesis  verification  and 
theory  formation  activities  in  the  domain  of  electronic 
troubleshooting.  Much  of  its  logical  or  inferencing 
capabilities  are  derived  from  uses  of  simulation  models 
in  conjunction  with  numerous  procedural  specialists.  The 
system  also  includes  a  highly  tuned  structural  parser  for 
allowing  the  student  to  communicate  in  natural  language. 
Students  can  test  components  in  an  electronic  circuit, 
develop  hypotheses  regarding  the  fault,  and  obtain 
feedback  concerning  why  a  hypothesis  is  incorrect. 
Replacement  of  parts  is  allowed  only  when  the  student  can 
answer  correctly  queries  about  why  he  thinks  the  parts 
should  be  replaced.  Although  the  system  is  extremely 
large  it  is  sufficiently  fast  to  be  thoroughly  exercised 
in  a  training  or  classroom  environment. 

22  pages,  11  references 
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Brown,  J.S.;  Rubinstein,  R. ;  Burton,  R.  (1976)  "Reactive 

Learning  Environment  for  Computer  Assisted  Electronics 

Instruction."  Lowry  Air  Force  Base,  CO:  AFHRL-TR-76-68. 

The  development  of  several  new  computer  based  strategies 
for  teaching  ,  troubleshooting  principles  to  electronics 
technicians  is  described.  The  report  documents  an 
experiment  in  which  those  materials  were  presented  to 
student  technicians  to  determine  their  attitudes  toward 
the  techniques,  and  to  determine  whether  the  resulting 
training  resulted  in  improvement  of  their  technical 
skills.  Results  in  both  cases  were  positive  with 

students  responding  very  favorably  to  the  materials  and 
with  their  performance  improving  quantitatively  and 
qualitatively  after  the  instruction. 

Seventeen  college  level  students  in  an  electronics 
curriculum  were  subjects  in  an  experiment  for  evaluating 
alternative  training  and  teaching  methodologies  on 
troubleshooting  behavior.  Subjects  were  divided  into 
four  groups,  each  group  receiving  a  different  sequence  of 
teaching  and  training  activities.  Most  of  the  evaluation 
is  qualitative  and  based  on  extensive  interviews  and 
questionnaires.  However,  the  fourth  group  was 

administered  a  pre-test  for  measuring  basic  understanding 
of  power  supply  troubleshooting.  This  pre-test  also 
included  two  faults  to  troubleshoot  before  the  laboratory 
games  began.  A  post-test  was  administered  to  measure  the 
effect  of  computer-based  activities  on  learning. 
Students'  scores  reflected  their  ability  to  make  correct 
fault  hypotheses  given  a  set  of  symptoms.  Improvement  in 
all  subjects'  scores  was  found  to  be  significant  at  the 
.05  level.  Also,  subjects  experienced  an  average 
decrease  in  task  time  by  41%,  significant  at  the  .05 
level . 

140  pages 
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Crawford,  A.M.;  Crawford,  K.S.  (1978)  "Simulation  of 
Operational  Equipment  with  a  Computer-Based  System:  A  Low  Cost 
Training  Technology."  Human  Factors  .  20 (2 ): 21 5-224. 

The  feasibility  and  effectiveness  of  teaching  performance 
skills  using  a  computer-based  training  (CBT)  methodology 
were  investigated.  Graphic  simulations  of  the  appearance 
and  functions  of  a  system  in  an  anti-submarine  aircraft 
were  presented  to  students  within  an  instructional 
framework.  The  objective  was  to  determine  whether  CBT 
could  be  used  for  low  cost,  part-task  training.  The 
subjects  were  new  co-pilot  trainees  for  the  S-3A 
anti-submarine  aircraft.  The  performance  of  the  22  CBT 
students,  as  measured  on  a  high  fidelity  simulator,  was 
compared  to  that  of  22  students  who  had  gone  through 
conventional  training  consisting  of  workbook  study  and 
hands-on  practice  in  the  simulator.  Results  showed  that 
CBT  students  performed  the  necessary  skills  as  well 
before  practice  in  the  simulator  as  conventionally 
trained  students  could  after  this  practice.  Performance 
measures  were  the  number  of  tasks  completed  and  the  time 
to  completion.  The  authors  felt  that  CBT  was  an 
effective  and  low-cost  means  of  part-task  training  of 
certain  performance  skills. 

10  pages,  11  references 


Crooks,  W.H.;  Kuppin ,  M.A.;  Freedy,  A.  (1977)  "Application  of 
Adaptive  Decision  Aiding  Systems  to  Computer-Assisted 
Instruction:  Adaptive  Computerized  Training  System  (ACTS)." 
Arlington,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences.  PATR-1 028-77-1 . 

This  report  describes  the  Adaptive  Computerized  Training 
System  (ACTS)  which  combines  the  techniques  of  circuit 
simulation,  artificial  intelligence,  decision  modeling, 
and  adaptive  computer-assisted  instruction  to  provide 
training  in  decision  making.  The  ACTS  incorporates  an 
adaptive  computer  program  which  learns  a  student's 
diagnostic  value  structure  and  uses  this  structure  to 
train  the  student  in  practical  decision  making.  This 
report  describes  the  development  and  operation  of  the 
ACTS  as  it  is  applied  to  training  electronics 
troubleshooting . 

Six  college  students  with  electronics  backgrounds  served 
as  subjects  in  solving  45  fault  diagnosis  problems. 
Subjects  were  divided  into  three  groups  receiving  the 
following  initial  aiding:  1)  the  expert's  three  best 
alternatives  were  displayed  to  the  subject  and  after  the 
subject's  choice  was  made,  additional  feedback  was 
provided  by  identifying  the  expert's  best  alternative; 

2)  subjects  first  indicated  alternatives  before  expert's 
alternatives  were  displayed;  and  3)  no  feedback  or 
indication  of  expert's  choice.  Results  indicate  that 
those  who  received  aiding  or  feedback  solved  problems  at 
lower  costs  than  subjects  who  received  neither  aiding  nor 
feedback.  Practice  was  shown  to  have  a  positive  effect 
for  all  groups  in  terms  of  reducing  costs. 
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Elliott,  T.K.  (1966)  "A  Comparison  of  Three  Methods  for 
Presenting  Procedural  Troubleshooting  Information.”  Aerospace 
Medical  Research  Laboratories,  Wright-Patter son  Air  Force  Base, 
Ohio.  Report  No.  AMRL-TR-66-1 91 • 

The  effects  of  subject  aptitude  and  performance  aide  mode 
of  presentation  on  the  performance  of  procedural, 
between-stage  troubleshooting  tasks  on  a  real  piece  of 
electronic  equipment  were  studied.  The  study  used 
nondecison  aids  presented  in  three  modes,  namely,  an 
automatic  retrieval  of  visual  information,  an  automatic 
retrieval  of  audio  information,  and  a  manual  retrieval  of 
visual  information.  Two  aptitude  groups  (Air  Force 
electronic  index  40-60  and  75-90)  with  no  previous 
electronic  training  or  experience  were  given  from  3  to  5 
hours  of  task  training  before  the  experiment.  Each 
subject  solved  26  actual,  and  11  synthetic  problems.  The 
study  indicated  no  difference  in  effectiveness  of  aids 
using  visual  representations.  Both  were  superior  to 
audio  presentations.  The  two  aptitude  groups  were 
equally  effective  in  performing  troubleshooting  tasks 
using  nondecision  aids.  In  comparison  with  a  previous 
study  using  decision  aids,  the  study  showed  nondecision 
aids  produced  superior  performance  on  the  same 
between-stage  troubleshooting  problems  using  similar 
subjects . 
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Two  groups  of  subjects  solved  the  same  set  of  13 
troubleshooting  and  repair  problems  in  seven 
solid-state-circuit  modules  which  contained  as  many  as 
five  stages  each.  Both  groups  used  the  same  hand  tools 
and  test  equipment.  One  group  was  composed  of  41 
conventionally  trained  Air  Force  5-  and  7-level 
technicians  who  normally  maintain  such  equipment  as  part 
of  their  jobs.  The  technicians  used  the  same 
troubleshooting  techniques  they  ordinarily  used  on  their 
jobs,  and  they  were  provided  with  a  performance  aid 
resembling  an  Air  Force  technical  order.  The  other  group 
was  composed  of  20  high-school  students  with  no  prior 
training  or  experience  in  electronics.  Their  training 
for  this  study  consisted  of  a  12-hour  course  in  the  use 
of  hand  tools  and  test  equipment  and  in  the  u§,e  of  the 
procedur alized  troubleshooting  aid  evaluated  in  this 
study.  The  aid  indicated  which  check  to  make  based  upon 
the  outcome  of  previous  checks.  Using  the  proceduralized 
troubleshooting  aid,  the  high-school  students  took 
significantly  less  troubleshooting  time  than  did  the 
experienced  technicians  using  normal  techniques; 
however,  the  technicians  require'd  significantly  less 
repair  time  and  made  significantly  fewer  errors  than  did 
the  students.  These  differences  may  be  accounted  for  in 
part  by  differences  in  the  tasks  and  scoring  methods  for 
the  two  groups,  and  there  were  no  differences  between  the 
two  groups  on  a  number  of  other  measures.  However,  the 
difference  in  training  time  and,  therefore,  cost  of 
training  between  the  two  groups  was  so  great  as  to 
suggest  the  possibility  that  job-relevant  training  and 
proceduralization  of  the  task  can  introduce  substantial 
savings,  even  after  the  cost  of  developing  the  special 
performance  aids  required  by  proceduralized 
troubleshooting  is  subtracted. 
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The  aim  of  this  study  was  to  improve  maintenance  of 
electronic  equipment  by  determining  what  information  to 
place  on  the  equipment  and  developing  techniques  for  its 
display.  Trips  to  various  communications  installations 
were  made  in  order  to  examine  equipment,  observe 
maintenance  activities,  examine  manuals,  and  interview 
personnel.  Detailed  data  were  collected  showing  the 
marked  variation  between  observed  test  point  readings  and 
those  called  for  in  maintenance  manuals.  Recommendations 
were  developed  concerning  information  to  be  displayed  on 
prime  electronic  equipment,  covering  1)  designation  of 
functional  groupings,  2)  identification  of  signal  paths, 

3)  identification  and  sequence  for  using  test  points,  and 

4)  presentation  of  historical  information  for  each  test 
point.  These  recommendations  were  evaluated  by 
incorporating  them  on  an  oscilloscope  and  evaluating  the 
performance  of  18  technicians.  Comparing  this  "coded" 
scope  with  an  identical  uncoded  one  showed  that 
troubleshooting  time  was  reduced  on  the  average  to 
approximately  one-half  that  for  the  uncoded  equipment. 
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This  study  used  a  commercial  real  equipment  simulator  to 
train  automechanics  for  fault  diagnosis.  The  research 
also  explored  the  attitude  of  the  students  and  the 
instructors  towards  the  simulator.  The  experimental 
group  received  simulator  practice,  while  the  control 
group  received  no  treatment.  Those  students  who 
practiced  on  the  simulator  had  better  troubleshooting 
performance  on  four  of  the  six  criteria  measures.  The 
instructors  were  satisfied  with  the  simulator,  and  the 
students  believed  that  it  was  at  least  as  good  as 
traditional  classroom  or  laboratory  instruction. 
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Finch,  C.R.  (1972)  "The  Effectiveness  of  Selected 
Self-Instructional  Approaches  in  Teaching  Diagnostic  Problem 
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This  study  examined  the  effects  of  three 
self-instructional  approaches  of  teaching  diagnostic 
problem-solving  (troubleshooting).  Two  of  the  approaches 
attempted  to  simulate  student-equipment  interaction  via 
programmed  and  text  instruction  while  a  third  approach 
utilized  actual  equipment.  After  forty-five  community 
college  students  received  an  orientation  to  equipment 
operation,  fifteen  were  randomly  assigned  to  each  of  the 
three  treatments.  Criterion  variables  included 
troubleshooting  knowledge,  troubleshooting  performance, 
and  attitude  toward  instruction  received.  Analysis  of 
covariance  indicated  that  the  equipment  approach  was 
superior  to  non-equipment  simulation  approaches  in  terms 
of  troubleshooting  performance  scores.  Correlational 
analyses  of  criterion  variables  provided  some  evidence 
that  relationships  between  attitude  and  behavior  were 
dependent  upon  the  instructional  approach  used. 
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The  report  discusses  a  system  for  Computerized  Decision 
Training  (CDT).  This  system  has  the  capability  to 
compare  subjects'  troubleshooting  decisions  with  the 
optimal  as  defined  by  "experts."  The  goal  was  to  train 
for  quality  decision  making  rather  than  cookbook 
troubleshooting.  The  simulations  are  computer  generated 
problems  applicable  to  electronic  circuits.  The  problems 
are  presented  on  a  CRT  and  permit  the  subject  to:  check 
symptoms,  measure  a  component  value,  replace  a  module,  or 
declare  circuit  to  be  fault  free.  The  adaptive  decision 
t  model  was  found  to  converge  on  the  decision  behavior  of 

the  students  who  quickly  settle  into  consistent  decision 
strategies.  The  authors’  conclusion  notes  that  there  is 
a  need  for  considerably  more  research  in  this  area. 
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Problem  Solving  Tasks."  Educational  and  Psychological 
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This  work  describes  a  method  of  fault  diagnosis  training 
relying  on  the  "tab  item."  The  tab  item  is  a  foldout 
cardboard  flow  chart  which  allows  the  troubleshooter  to 
gather  information  by  pulling  tabs.  This  system  provides 
the  following:  A  symptom,  check  procedures,  and  a  list 
of  possible  failures.  This  system  therefore  attmepts  to 
reproduce  the  troubleshooting  situation  on  paper. 
Measures  for  scoring  troubleshooting  performance  which 
are  discussed  are:  number  of  tests  performed,  number  of 
redundant  tests,  test  relevancy,  and  test  adequacy. 
Although  this  article  offers  no  empirical  evidence  of  the 
instructional  value  of  the  tab  item,  it  does  speculate  as 
to  its  potential  value  across  a  broad  spectrum  of 
technical  training.  Additional  uses  of  the  Tab  Test  can 
be  found  in  Standlee  (1956)  and  Fattu  (1956). 
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Glass,  A. A.  (1967)  "Problem-Solving  Techniques  and 

Troubleshooting  Simulators  in  Training  Electronic  Repairmen." 

Doctoral  Thesis.  Columbia  University. 

The  purpose  of  this  project  was  to  study  the  practical 
and  operational  application  of  problem-solving  concepts 
by  introducing  into  the  regular  training  of  military 
electronic  repairmen  two  experimental  methods  of  teaching 
troubleshooting  and  to  evaluate  their  effectiveness  in 
promoting  success.  These  two  experimental  methods  were: 

1.  a  series  of  recorded  lectures  dealing  with  problem 
solving  methods,  and  2.  a  series  of  paper  and  pencil 
troubleshooting  simulators  to  provide  the  opportunity  to 
apply  problem  solving  techniques.  There  were  two 
simulators,  one  commercial  and  the  other  custom-built. 

The  main  criteria  for  evaluation  of  the  experimental 
variables  were  performance  tests  on  actual  electronic 
equipment  (N=24Q).  The  method  for  administering  and 
scoring  the  performance  tests  was  the  result  of  extensive 
pilot  studies.  The  essence  of  the  scoring  method  was  to 
score  for  sequence  of  logic  in  troubleshooting,  time,  and 
solution  success.  The  conclusions  were:  1.  Subjects 
receiving  problem-solving  lectures  were  superior  on 
performance  test  to  subjects  not  receiving  them.  2. 
Subjects  using  simulators  were  superior  on  performance 
criteria  tests  to  those  not  using  the  devices,  but  only 
when  the  custom-built  device  was  used.  3-  Subjects 
receiving  a  combination  of  lectures  and  devices  were 
superior  on  performance  criteria  tests  to  subjects  not 
receiving  the  combination.  4.  The  use  of  the 
custom-built  simulator  produced  superior  performance 
criteria  test  scores  as  compared  to  subjects  using  the 
commercial  version,  only  when  the  lectures  were  used  as 
well. 
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An  experiment  was  designed  (1)  to  test  the  hypothesis 
that  the  half-split  instruction  increased  efficiency  for 
simple  problems  and  (2)  to  determine  half-split 
efficiency  for  more  .  complex  problems.  A  second 
experiment  was  designed  to  determine  whether  additional 
practice,  -different  methods  of  instruction,  and  more 
complex  training  materials  would  produce  an  improvement 
in  the  subject's  ability  to  arrive  at  a  set  of 
alternatives,  and,  if  so,  whether  the  half-split  method 
would  then  be  more  efficient. 

In  experiment  I,  90  male  university  students  participated 
as  subjects  and  were  instructed  to  replace  a  faulty  unit 
in  a  troubleshooting  task.  Thirty  were  assigned  problems 
to  each  of  three  complexity  levels.  Based  on  scores 
obtained  from  a  problem  solving  test,  subjects  were 
randomly  assigned  to  a  control  group  and  a  group  which 
received  explicit  instructions  regarding  the  application 
of  the  half-split  technique.  Solution  time  for  the 
half-split  group  was  significantly  longer  than  the 
control  group.  Increasing  complexity  resulted  in  an 
increase  in  the  number  of  tests  only  for  those  subjects 
who  scored  low  on  the  problem  solving  test.  Results 
suggest  that  the  half-split  technique  is  superior  under 
certain  conditions,  such  as  with  diagrams  of  low 
complexity  where  subjects  apparently  had  little 
difficulty  in  finding  the  trouble-possible  units,  and 
where  subjects  avoided  incorrect  replacements. 

Experiment  II  was  an  attempt  to  improve  the  effectiveness 
of  the  half-split  method  by  improving  instruction  and 
giving  extended  practice.  Only  problems  of  the  highest 
level  of  complexity  were  used.  Forty  male . university 
students  participated  and  were  randomly  assigned  to  four 
training  groups.  The  deductive  half-split  group  was  like 
the  previous  half-split  group  but  given  more  practice. 

The  instructions  and  practice  given  the  deductive 
half-split  group  were  more  effective  in  reducing  the 
number  of  tests  than  were  the  less  extensive  instructions 
in  Experiment  I.  Results  of  both  experiments  indicate 
that  the  empirical  superiority  of  the  half-split 
technique  is  a  function  of  the  nature  of  the  task,  and 
the  degree  and  type  of  training  given  the  troubleshooter. 

From  these  experiments,  it  is  now  clear  that  future 
research  on  training  methods  should  emphasize  the  more 
basic  deductive  phase  rather  than  the  relatively 
mechanical  half-splitting  aspect  of  the  process. 
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Operation  (P0G0)  was  used  in  the  design,  implementation 
and  testing  of  a  computer-assisted  instruction  course  to 
train  airmen  in  malfunction  diagnosis  -  CATTS  (Computer 
Aided  Training  in  Troubleshooting).  The  design  of  the 
course  attempted  to  reduce  the  problems  of  computer 
graphics  for  both  instructor  ana  student.  The 
observations  of  Air  Force  instructors  using  the  system 
suggest  that  the  system  is  workable.  No  empirical 
evidence  was  gathered.  The  report  emphasizes  the  POGO 
software  more  than  troubleshooting. 
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The  hypothesis  formulated  was  that  troubleshooting 
performance  on  simulation  would  be  predictive  of  the  same 
tasks  on  real  equipment.  Sixteen  instructors  and  fifteen 
beginning  students  in  a  precision  measurement  equipment 
course  were  given  the  same  problems  with  two  treatment 
levels.  Problems  were  solved  for  criterion  of  job  sample 
test  (JST)  and  simulation  or  symbolic  performance  test 
(SPT).  The  task  in  each  case  was  to  locate  the  fault  at 
the  stage  level  rather  than  component  level.  Performance 
measures  included  mean  percent  correct  solution,  time  to 
complete  problems,  and  number  of  steps  to  completion. 
Statistically  significant  correlations  indicate  a  high 
degree  of  similarity  between  the  job  sample  test  and 
symbolic  performance  test.  An  algorithm  is  presented 
which  is  used  to  evaluate  problem  difficulty. 
Correlation  of  the  hypothesis  problem  difficulty  with 
empirical  problem  difficulty  (obtained  by  using  test 
results  of  subjects)  did  not  produce  significant 
agreement.  It  was  noted  that  the  problem  difficulty 
algorithm  was  insensitive  to  feedback  loops  which 
occurred  in  some  of  the  problems  and  would  be  modified  in 
the  future. 
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Ss'  capabilities  for  processing  probability  information 
in  performing  a  diagnostic  search  task  similar  to 
troubleshooting  is  explored.  Six  Ss  searched  (queried) 
among  20  components  in  a  series  circuit  to  isolate  a 
single  component  failure  per  trial.  Failures  were 
generated  according  to  10  alternative 

probability-of-component-failure  distributions  (D),  each 
N  =  500.  Prior  to  performing  their  search  on  a  trial,  Ss 
predicted  the  failure  for  that  trial.  Of  primary 
interest  was  the  influence  of  varying  D  on  predictions 
and  search  performance.  Results  indicate  that:  (1) 

distributions  of  predictions  tend  'to  match  actual  D  in 
shape  and  location  but  exhibit  certain  characteristics; 

(2)  probability  of  correct  prediction  is  low  and  is 
influenced  by  D;  (3)  search  strategies  are  suboptimal 
(The  optimal  solution  required  an  average  of  14%  fewer 
tests.)  but  "logical"  and  to  some  extent  predictable; 
and  (4)  in  some  instances,  search  strategy  appears  to  be 
related  to  predictions. 
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This  paper  describes  an  approach  to  the  selection  of 
inexpensive  alternatives  to  fully  job-relevant 
performance  tests.  Based  on  the  concept  of  content 
validity,  the  approach  is  to  utilize  the  skill  and 
knowledge  required  of  the  trainee  as  a  criterion  for 
assessing  the  cost  and  job-relevance  of  fabricated 
alternatives  to  performance  tests.  Two  job  tasks  are 
analyzed  to  illustrate  the  concept. 
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An  experiment  was  conducted  at  a  nuclear  research 
institute  to  investigate  troubleshooting  approaches  used 
by  skilled  electronics  repairmen  during  on-the-job  tasks. 

A  verbal  record  of  the  repairmen's  strategies  was 
collected  for  data  analysis.  Codes  were  applied  to 
describe  the  heuristics  employed:  1)  topographic  search 
-  use  the  topographic  location  of  measuring  points;  2) 
functional  search  -  use  the  normal  functional  relation 
between  a  feature  in  the  system's  response  and  a  specific 
part  in  the  system;  and  3)  search  by  evaluation  of  a 
fault.  This  last  heuristic  implies  the  human  is  using  a 
mental  model  of  the  system  relating  changes  to  internal 
signals,  parts  or  components  to  the  changes  observed  in 
the  system  response.  Results  suggest  that  procedures 
considered  unsystematic  in  a  laboratory  study  may  turn 
out  to  be  rational  and  systematic  when  one  listens  to  the 
individual's  rationalization.  Also,  it  was  found  that 
verbal  records  appear  to  be  useful  for  recording  rapid 
sequences  of  simple  decisions  based  upon  informationally 
redundant  observations . 
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In  the  report  are  discussed  some  aspects  of  state 
identification  and  diagnosis  in  process  plant  control 
which  must  be  considered  in  connection  with  automatic 
disturbance  analysis  and  man-machine  interface  systems. 

The  content  of  the  diagnostic  process  depends  upon  the 
overall  goal  of  the  diagnostician  -  whether  it  is  to 
protect  the  plant,  to  maintain  operation  or  to  repair, 
and  it  may  not  necessarily  include  a  determination  of  the 
cause  itself.  Important  aspects  of  diagnosis  include 
critical  variables  and  causal  flow  paths  which  are 
intimately  related  to  the  paths  along  which  events  and 
changes  propagate  through  the  system,  i.e.  to  the  flow 
of  energy,  matter  and  information  which  together  form 
complex,  interacting  flow  structures. 

In  the  process  plant  environment,  a  diagnostic  task 
implies  a  search  to  identify  a  change  from  a  normal  or 
planned  plant  state.  Several  elementary  strategies  can 
be  identified.  In  the  report  a  distinction  is  drawn 
between  two  main  groups  -  topographic  search  strategies , 
performed  as  search  through  the  system  with  reference  to 
a  model  of  normal  plant  state;  and  symptomatic  search 
strategies ,  performed  as  a  search  through  a  library  of 
abnormal  state  models  with  reference  to  the  actual  plant 
state . 

Typical  properties  of  the  different  strategies  are 
discussed  such  as  processing  capacity  requirements  and 
dependence  upon  a  priori  analysis. 

The  role  of  the  elementary  strategies  in  the  overall 
diagnostic  task  is  not  discussed  since  this  is  considered 
possible  only  with  reference  to  selected  scenarios 
describing  realistic  and  complex  real  life  situations 
arising  from  disturbed  plant  operations.  The  aim  of  the 
report  is  to  identify  and  formulate  some  of  the  issues 
which  should  be  taken  into  account  when  creating  such 
scenarios  based  on  careful  field  studies  and  analysis  of 
incident  reports.  Based  on  these  scenarios,  different 
overall  diagnostic  strategies  can  then  be  formulated  and 
tested  by  simulator  experiments. 
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The  elements  of  an  instructional  system  are  discussed  and. 
some  literature  bearing  on  the  functions  of  these 
elements  is  reviewed,  with  the  objectives  of  stimulating 
thought  about  an  instructional  system  as  a  CAI  test-bed 
and  of  pointing  out  some  results  of  laboratory  research, 
particularly  in  cognitive  psychology,  that  are  believed 
to  be  worthy  of  the  instructional  technologist's 
attention . 

In  the  second  part  of  the  report,  selected  parts  of  CAI 
projects  underway  at  the  Behavioral  Technology 
Laboratories  are  described  in  relation  to  instructional 
system  elements,  to  give  an  overview  of  this  research. 

The  projects  include:  work  on  computer  grahics  to  (1) 
simulate  front-panel  topography  and  functional 
organization  of  circuits,"  (2)  facilitate  interpretation 
of  relative  motion,  and  (3)  create  visual  analogies  for 
abstract  concepts  and  processes  to  serve  as  the  basis  for 
students  to  develop  their  own  internal  representations; 
work  on  new  methods  for  recording  cortical  evoked 
potentials  and  correlating  them  with  learning  and  memory 
processes;  work  on  a  dynamic  programming  model  for 
adaptive  control  of  problem-solving  types  of  CAI;  and 
work  on  three-dimensional  mock-ups  of  electronic 
equipment  to  be  placed  on-line  with  a  CAI  system. 
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Three  computer-based  techniques  for  analyzing  and 
simulating  serial-action  tasks  are  described.  The  first, 
called  BETS  (a  Bayesian  model),  measured  the  efficiency, 
in  terms  of  -expected  information ,  of  tests  made  by 
technicians  who  -were  troubleshooting  malfunctioning 
electronic  circuits.  It  computed  efficiency  ratios  for  a 
technician's  test  sequences  in  comparison  to  optimal 
sequences.  The  second  method,  ARMAN,  generated  detailed 
time-and-motion  analyses  from  gross  descriptions  of 
serial-action  tasks  and  man-machine  interfaces  and 
computed  the  time  costs  of  these  tasks. 

The  third  technique  incorporates  a  general  model  of  the 
action-goal  structure  of  serial-action  work.  It  operates 
on  this  structure  with  special  functions  composed  in  a 
symbol  manipulation  language.  This  program,  called 
TASKSYM,  can  generate  all  alterna-tive  correct  ways  to 
accomplish  serial-action  work  and  can  track  a  subject 
through  the  performance  of  this  work.  The  model  includes 
an  anti-goal  structure  which  identifies  action  sequences 
leading  to  catastrophic  error,  e.g.,  electrical  damage  to 
the  equipment  or  shock  hazard  to  the  performer.  TASKSYM 
is  extended  with  LISP  to  develop  TASKTEACH  which  provides 
feedback  on  tests  and  errors,  and  offers  various  levels 
of  help. 
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A  method  for  using  a  computer  time-sharing  system  to 
assist  the  learning  of  serial  tasks,  from  operating 
equipment  through  troubleshooting,  is  described.  The 
method  is  based  on  mediational  theory.  The  current 
resurgence  of  interest  in  mediational  theory  in 
psychological  research  is  noted,  and  parallels  between 
experimental  strategies  and  instructional  strategies  for 
evoking  and  manipulating  mediating  processes  are  pointed 
out.  Categories  of  processes  which  mediate  performance 
of  serial  tasks  are  described.  Procedures  for 

facilitating  the  learning  of  these  processes  are 
implemented  by  a  computer  program,  called  TASKTEACH.  The 
program  sustains  the  student's  performance  of  complex 
serial  tasks  by  giving  him  variable  amounts  of  support 
while  helping  him  learn  and  organize  the  processes  which 
mediate  his  performance  of  these  tasks.  The  program 
generates  output  to  the  student  during  the  learning 
session  by  processing  short  lists  and  the  prior  responses 
of  the  student.  The  lists,  which  are  input  to  the 
program  replace  the  conventional  frame-by-frame 
description  of  an  instructional  sequence  written  in  a 
"CAI  language.” 
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The  Generalized  Maintenance  Trainer-Simulator  (GMTS)  is  a 
concept  for  giving  students  in  Class  C  schools  intensive 
practice  in  troubleshooting  equipment  and  systems  taught 
in  those  schools.  It  can  be  used  for  any  device  in  which 
signal  paths  and  their  relationships  to  controls, 
indicators,  and  test  points  can  be  defined.  The  GMTS 
uses  generative  CAI.  That  is,  it  generates  the 
interaction  with  the  student  by  referring  to  his  last 
inputs  and  to  its  stored  history  of  interactions  with  him 
up  to  that  point.  To  the  extent  that  these  individual 
student  histories  constitute  models  of  individual 
students,  GMTS  constructs  a  model  of  each  student  to 
interact  with  that  student.  In  addition  to  these 
features,  GMTS  is  uniquely  suitable  for  use  in  Class  C 
school  training.  The  computer  program  that  implements 
the  instructional  system  is  indifferent  to  the  specific 
equipment  being  taught.  What  specific  equipment  that  is 
simulated  by  the  GMTS  is  determined  by  loading  two  data 
bases  for  that  equipment:  one  describing  essential 
internal  features  of  the  equipment  and  the  other 
describing  the  external  features. 

This  is  a  report  of  a  field  evaluation  of  the  GMTS 
applied  to  systems  level  troubleshooting  in  the  UHF 
communications  side  of  the  Fleet  Communications  System. 
Twenty  Class  A  school  students  waiting  to  enter  C  schools 
practiced  solving  thirty-five  troubleshooting  ‘  problems. 
Results  were  generally  positive.  With  respect  to 
practice,  the  students  became  uniformly  more  fluent  at 
troubleshooting;  mean  times  to  solve  a  problem  were 
decreased  by  a  factor  of  two,. and  standard  deviations  of 
these  time  were  decreased  by  a  factor  of  five.  Similar 
reductions  existed'  for  average  number  of  extra 
replacements  per  problem.  Also,  students’  attitudes 
toward  the  trainer  were  generally  favorable.  An 
additional  field  trial  is  underway,  using  the  AN/SPA-66 
radar  repeater  as  the  subject  matter. 
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Field  Maintenance  Interface  Between  Human  Engineering  and 
Maintainability  Engineering.”  Human  Factors.  1 2  C  3 ) : 253-259  - 

Field  maintenance  actions  reported  in  a  deployed  air 
defense  system  were  reviewed  to  determine  and  quantify 
(1)  the  nature  of  operator  and  maintainer  influences  on 
system  malfunctions,  and  (2)  the  nature  of  man-machine 
problems  being  encountered  by  system  personnel  in  actual 
diagnosis  and  repair  activities.  Maintenance  events 
described  in  213  problem  reports  from  the  field  were 
categorized  according  to  three  probable  sources  of 
malfunction:  primarily  human,  combination  human  and 
equipment,  and  primarily  equipment.  Human  involvement  in 
system  malfunctions  was  clearly  established  in  25%  of  the 
events  reported.  Additional  information  contained  in  21 
documents  analyzing  system  maintenance  was  examined.  The 
nature  of  human  influence  on  system  malfunction  is 
described  by  examples  of  "unfortunate"  field  maintenance, 
"ordinary"  field  maintenance,  and  troubleshooting 
deficiencies.  Troubleshooting  deficiencies  were  cited  as 
a  frequent  problem  observed  by  equipment  contractors  who 
had  more  knowledge  and  understanding  than  the  technical 
personnel.  The  nature  of  reported  man-machine  problems 
highlighted  the  fact  that  human  engineering  participation 
in  establishing  maintainability  design  requirements  can 
serve  to  make  equipment  that  will  be  suitable  to  the 
presence  and  actions  of  human  users. 
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Cybernetics .  SMC-8 (4 ) : 258-27X1 

It  is  proposed  that  humans  in  automated  systems  will  be 
asked  to  assume  the  role  of  troubleshooter  or  problem 
solver  and  that  problems  which  they  will  be  asked  to 
solve  in  such  systems  will  not  be  amenable  to  rote 
solution.  The  design  of  visual  displays  for  problem 
solving  in  such  situations  is  considered,  and  the  results 
of  two  experimental  investigations  (N=20)  of  human 
problem  solving  performance  in  the  diagnosis  of  faults  in 
graphically  displayed  network  problems  are  discussed. 

The  effects  of  problem  size  (9, 25, and  49  components), 
forced-pacing,  computer  aiding,  and  training  are 
considered.  Results  indicate  that  human  performance 
deviates  from  optimality  as  problem  size  increases. 
Forced-pacing  appears  to  cause  the  human  to  adopt  fairly 
brute  force  strategies,  as  compared  to  those  adopted  in 
self-paced  situations.  Computer  aiding  substantially 
lessens  the  number  of  mistaken  diagnoses  by  performing 
the  bookkeeping  portions  of  the  task.  Further,  for 
self-paced  situations,  skills  developed  with 
computer-aided  training  are  transferred  to  unaided 
situations . 
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Diagnosis  Task."  IEEE  Transactions  on  Systems,  Man,  and 
Cybernetics.  SMC-8 (577157-351: 

Utilizing  elementary  concepts  from  the  theory  of  fuzzy 
sets  as  well  as  several  nonfuzzy  heuristics,  a  model  is 
presented  of  human  decisionmaking  in  the  task  of 
troubleshooting  graphically  displayed  networks.  The 
performance  of  the  model  is  compared  to  the  results  of 
two  previously  reported  experimental  studies.  The 
ability  of  the  model  to  represent  human  decision  making 
as  a  function  of  network  size,  forced-pacing,  and 
computer  aiding  is  considered.  The  model  is  used  to 
support  the  hypothesis  that  humans  tend  to  discount  the 
value  of  information  about  what  has  not  failed  in 
searching  for  the  sources  of  malfunctions. 
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A  previously,  reported  model  of  human  decision  making  in 
fault  diagnosis  tasks  is  extended  to  include  situations 
where  feedback  and  component  redundancy  are  important 
considerations.  The  model  is  based  on  concepts  from  the 
theory  of  fuzzy  sets.  The  results  of  an  experiment  with 
human  subjects  are  reported  and  used  to  estimate  the 
parameters  of  the  model.  The  model  is  used  to  illustrate 
the  notion  that  subjects  who  discount  the  importance  of 
feedback  loops  tend  to  have  considerable  difficulty  when 
they  encounter  problems  having  a, large  number  of  loops. 
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Trainees  in  a  Fault  Diagnosis  Task."  Human  Factors. 

21  (2):  195-203. 

Forty  trainees  in  an  FAA  certificate  program  participated 
in  an  experimental  study  of  troubleshooting  of 
graphically  displayed  networks.  The  effects  of  network 
size,  computer  aiding,  and  training  were  considered.  It 
was  found  that  performance  degraded  as  network  size 
increased,  improved  with  the  use  of  computer  aiding,  and 
that  skills  developed  with  computer  aiding  were 
transferred  to  the  unaided  situation. 
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Rouse,  W.B.  (1979)  "Problem  Solving  Performance  of  First 
Semester  Maintenance  Trainees  in  Two  Fault  Diagnosis  Tasks." 
Human  Factors .  21(5). 

Forty-eight  first  semester  trainees  in  an  FAA  certificate 
program  participated  in  an  experimental  study  of 
troubleshooting  of  two  different  types  of  graphically 
displayed  networks.  The  effects  of  network  size,  redundancy, 
feedback,  computer  aiding,  and  training  were  considered.  It 
was  found  that  performance  degraded  as  network  size 
increased,  degraded  as  the  level  of  feedback  was  reduced, 
improved  with  the  use  of  computer  aiding,  and  that  skills 
developed  with  computer  aiding  in  one  task  were  transferred 
to  the  other  task. 
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Rouse,  W.B.;  Rouse,  S.H.  (1979)  "Measures  of  Complexity  of 
Fault  Diagnosis  Tasks."  to  appear  in  IEEE  Transactions  on  System, 
Man ,  and  Cybernetics,  SMC-9(11). 

The  literature  of  complexity  is  reviewed  and  the 
distinction  between  perceptual  complexity  and  problem 
solving  complexity  is  discussed.  Within  the  context  of 
two  particular  fault  diagnosis  tasks,  four  measures  of 
complexity  are  considered.  These  measures  are  evaluated 
using  data  from  two  previously  reported  experiments  which 
employed  eighty-eight  subjects.  It  is  shown  that  two 
particular  measures  of  complexity,  one  based  on 
information  theory  and  the  other  base'd  on  the  number  of 
relevant  relationships  within  the  problem,  are  reasonably 
good  predictors  of  human  performance  in  fault  diagnosis 
tasks.  The  success  of  these  measures  is  explained  by  the 
fact  that  they  incorporate  the  human’s  understanding  of 
the  problem  and  specific  solution  strategy  as  well  as 
properties  of  the  problem  itself. 
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Troubleshooting .  Technical  Report  53-2.  Lackland  Air  Force 
Base,  Texas:  Human  Resources  Research  Center. 

Troubleshooting  was  investigated  using  radar, 
reciprocating  engines,  and  remote  control  turrets  as  the 
hardware  for  malfunction  analysis'  observation.  The 
discrete  types  of  equipment  were  chosen  in  art  effort  to 
promote  generalizability .  The  following  hypotheses  were 
formulated  and  tested: 

1 .  Good  troubleshooters  know  more  about  the 
functioning  of  the  equipment  upon  which  they 
work  than  do  poor  troubleshooters . 

2.  Good  and  poor  troubleshooters  differ  in 
previous  experience. 

3.  Good  and  poor  troubleshooters  differ  in 
intelligence . 
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Saupe,  J.L.  (1954)  Troubleshooting  Electronic  Equipment :  An 

Empirical  Approach  to  the  Identification  of  Certain  Requirements 
of  a  Maintenance  Occupation .  Urbana,  ~lLl  Oniversity  oT 
TTlinois . 

The  study  attempts  to  identify  the  criteria  for 
proficiency  in  a  troubleshooting  situation.  The  study 
provides  "leads  for  the  construction  cf  proficiency  tests 
which  are  important  aspects  of  the  efficient  selection, 
training,  and  distribution  of  maintenance  personnel."  The 
goals  of  the  research  were: 

1.  to  develop  a  psychological  model  by  which 
to  view  troubleshooting  behavior. 

2.  to  test  certain  hypotheses  taken  from  this  model 
relative  to  aspects  of  the  troubleshooting 
process 

3*  to  illustrate  an  empirical  approach  to  the 
identif ication  of  important  objectives 
of  training  programs 

Subject  performance  was  evaluated  on  a  simulated  radio 
receiver  and  on  a  paper  and  pencil  test.  Empirical 
results  were  presented. 


Standlee,  L.S.;  Bilinski,  C.  (1966)  "Note  on  Simulated  Vs. 
Actual  Electronic  Troubleshooting  Performance."  Perceptual  and 
Motor  Skills.  23:532. 

This  is  a  brief  note  on  electronic  troubleshooting 
performance.  It  discusses  the  two  basic  approaches  to 
measuring  this  performance  (1)  measurement  of  performance 
on  simulated  and  (2)  actual  electronic  equipment.  114 
subjects  (Naval  electronics  technicians)  were  divided 
into  two  groups,  one  group  troubleshooting  with  the 
simulator,  and  the  other  with  actual  equipment.  Results 
indicated  poor  troublshooting  performance  with  the 
simulator  and  proficient  performance  with  the  actual 
equipment . 
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Steinemann,  J.H.,  (1966)  "Comparison  of  Performance  on  Analogous 
Simulated  and  Actual  Troubleshooting  Tasks."  U.S.  Naval 
Personnel  Research  Activity,  Memorandum  SRM  67-1,  July  1966. 

This  investigation  compared  the  performance  of  a  group  of 
subjects  assessed  on  a  simulated  troubleshooting  task  and 
on  the  identical  actual  troubleshooting  task  using  real 
equipment.  Subjects  were  14  students  in  the  experimental 
training  program  for  Electronics  Technicians,  conducted 
by  the  Navy  Training  Research  Laboratory,  San  Diego. 

Analysis  of  results  revealed  that  the  simulated 
performance  measure  did  not  provide  a  valid  estimate  of 
performance  proficiency  on  the  actual  task.  Obtained 
negative  inter-test  correlations  indicate  that  simulated 
test  results  would  actually  be  misleading  in  terms  of 
estimating  actual  performance  scores.  In  addition  to 
performance  score  discrepancies,  there  were  observable 
differences  in  specific  performance  procedures  and 
overall  troubleshooting  strategy  attributable  to  the 
differences  in  test  mode.  Although  the  research  design 
was  weak,  the  evidence  strongly  suggests  caution  in 
assuming  that  a  simulated  performance  measure,  even  with 
considerable  face  validity,  will  provide  a  valid  estimate 
of  actual  performance  on  a  common  task. 
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'•The  Efficient  Course  of  Action  in  'Troubleshooting'  as  a  Joint 
Function  of  Probability  and  Cost."  Educational  and  Psychological 
Measurement,  15 (4 ): 462-477 . 

This  research  focuses  on  the  development  of  a 
probabilistic  model  of  troubleshooting  behavior  which 
prescribes  an  efficient  course  of  action  in  locating 
defects  based  on  probabilities  of  failures  and  work  time 
given  particular  symptoms.  Data  were  obtained  from 
maintenance  records  for  46  four-engine  aircraft  to  obtain 
the  probability  of  each  possible  cause  of  a  particular 
set  of  diagnostic  signs.  Mean  repair  times  were  also 
completed  for  each  type  of  repair.  A  complete  method  and 
a  simplified  method  are  described  for  determining  an 
efficient  course  of  action  to  be  followed  in  reducing  a 
set  of  possible  causes  of  malfunction  to  the  actual 
cause.  The  simplified  method  was  proposed  as  an 
alternative  to  overcome  the  cumbersome  calculations  of 
the  complete  method.  The  simplified  method  uses  the 
ratio  of  mean  repair  time  to  probability  of  failure  to 
determine  the  course  of  action  to  follow  in  reducing  the 
list  of  possible  troubles.  Results  of  the  analyses  of 
maintenance  records  were  sufficiently  encouraging  to 
suggest  that  further  research  of  this  type  be  done  for  a 
variety  of  complex  devices. 
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Third  Biennial  Maintenance  Training  and  Aiding  Conference. 
Orlando,  Fla. 

A  responsive  and  intelligent  device  which  can  provide 
both  training  information  and  job  performance  aiding  can 
be  extremely  efficient  in  utilization  of  hardware, 
software,  and  so-called  course-ware.  The  hardware 
components  useful  for  presenting  automated  instruction 
are  essentially  the  same  as-  those  required  to  aid  an 
operator  or  technician,  viz.,  a  versitile  and  fast 
microprocessor  CPU,  ample  random  access  memory,  ample 
bulk  storage,  and  appropriate  input-output  media.  While 
the  software  driving  such  a  device  may  be  a  collection  of 
relatively  special-purpose  routines  designed  to  satisfy 
particular  training  or  aiding  needs,  a  large  portion  of 
this  software  may  be  common  utility  functions  which 
respond  to  specific  requests  of  the  user  or  serve  to 
manipulate  data  about  the  target  equipment  or  data  about 
the  operator /technician .  Finally,  the  data  base  which 
characterizes  a  target  equipment  can  be  designed  in  such 
a  way  that  both  training  and  aiding  requirements  are  met 
without  duplication  or  inefficiency  of  preparation 
effort. 

Under  a  contract  with  Naval  Training  Equipment  Center, 
Behavioral  Technology  Laboratories  has  assembled  the 
hardware  elements  of  such  a  system,  designed  the  target 
equipment,  and  programmed  routines  for  training  novice 
operator/technicians  and  aiding  equipment  operation  and 
maintenance . 


21  pages,  5  references 


Page  32 


Wescourt,  K.T.;  Hemphill,  L.  (1978)  "Representing  and  Teaching 

Knowledge  for  Troubleshooting/Debugging."  Institute  for 

Mathematical  Studies  in  the  Social  Sciences,  Report  No.  292, 

Stanford  University,  Stanford,  CA. 

An  extensive  review  of  the  troubleshooting/debugging 
literature  is  covered  from  an  artificial  intelligence 
perspective.  The  authors  were  interested  in  measuring 
the  effect  of  training,  that  is,  exposure  to  specific 
debugging  heuristics,  on  problem  solving  performance. 
Twelve  subject's  with  relatively  little  programming 
experience  participated.  They  were  currently  enrolled  in 
a  BASIC  programming  course.  Subjects  were  randomly 
assigned  to  two  groups  (aided  and  unaided)  based  on  a 
computer  programmer  aptitude  battery.  Aided  subjects 
could  refer  to  a  text  tutorial  during  problem  solving 
sessions.  The  chronology  of  subjects'  actions  were 
judged  according  to  5  criteria  relating  to  implementation 
of  heuristics  contained  in  the  tutorial  guidelines  and 
success  of  attempted  solution.  Successful  debugging 
grammars  that  would  capture  the  subjects'  strategies  were 
not  developed.  Subjective  judgments  of  participants  on 
the  research  team  were  necessary  to  identify  the  extent 
to  which  each  measure  applied  to  subjects'  strategies. 
Results  indicated  poor  performance  of  both  groups  in 
solving  problems.  After  the  experiment  large  differences 
were  found  in -post  test  scores  attained  on  the  computer 
programmer  aptitude,  battery,  and  thus  by  chance,  subjects 
assigned  to  the  non-aided  group  had  become  better 
programmers  since  completing  their  course  in  BASIC.  In 
conclusion,  there  were  no  strong  results  to  show  that  the 
tutorial  aid  affected  implementation  of  heuristics.  It 
is  suggested  that  a  pedagogy  is  needed  for  teaching 
procedural  rather  than  declarative  knowledge. 
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Czeh,  R.S.  (1957)  "Studies  of  Complex  Behavior  and  their 
Relation  to  Troubleshooting  in  Electronic  Equipment."  Alexandria, 
VA:  Human  Resources  Research  Organization . 

This  report  is  a  review  of  the  literature  concerned 
either  directly  or  .indirectly  with  the  troubleshooting  of 
electronic  equipment.  Specifically,  this  review  presents 
the  most  pertinent  findings  in  the  areas  of  problem 
solving  and  concept  formation  and  indicates  the  manner  in 
which  these  findings  have  impinged  upon,  or  could  impinge 
upon  thinking  about  troubleshooting.  The  literature  on 
troubleshooting  is  discussed  and  the  final  section 
outlines  a  research  task  designed  to  develop  an 
efficient,  generally  applicable  procedure  for 
troubleshooting  electronic,  and  to  develop  methods  for 
teaching  the  procedure. 

23  pages,  59  references 


Edwards,  J.;  Norton,  S. ;  Taylor,  S.  Weiss,  M.;  Dusseldorp,  R. 
(November  1975)  "How  Effective  is  CAI?  A  Review  of  the 
Research."  Educational  Leadership ,  33:  147-153. 

This  article  summarizes  many  research  studies  that 
involved  Computer  Assisted  Instruction.  The  CAI  modes 
discussed  are:  drill  and  practice,  problem-solving, 

simulation,  and  tutorial.  The  areas  of  CAI  research 
discussed  are: 

1.  CAI  as  a  summplement  to  traditional  instruction 

2.  CAI  as  a  substitute  for  traditional  instruction 

3.  Effectiveness  of  different  modes  of  CAI 

4.  Comparison  of  CMI  with  other  nontraditional  methods 
of  instruction 

5.  Compression  of  time 

6.  Retention  of  learning  with  CAI 

7.  Effectiveness  according  to  ability  levels 

This  literature  review  referred  to  9  studies  that  showed 
that  the  combination  of  CAI  and  traditional  instruction 
was  distinctly  better  than  traditional  instruction  only. 

As  a  substitute  for  traditional  instruction,  the  findings 
were  mixed  with  9  studies  finding  CAI  superior  and  8 
studies  finding  traditional  instruction  better.  In  all 
cases  CAI  was  effective  in  reducing  training  time. 
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Fattu,  N.A.  (1956)  A  Catalog  of  Troubleshooting  Tests .  Indiana 
University:  Institute  of  Educational  Research. 

This  catalog  contains  examples  of  various  kinds  of 
troubleshooting  tests.  The  justification  for  a  catalog 
of  troubleshooting  tests  is  that  it  summarizes  and 
illustrates  materials  assembled  from  widely  scattered 
sources  and  makes  them  available  to  individuals  who  need 
the  information  but  do  not  have  time  to  assemble  it. 

This  catalog  attempts  to  be  repr esentative  rather  than 
exhaustive . 

Tests  are  reported  in  the  following  groups: 

1.  on-the  job  measures 

2.  performance  tests 

3.  simulator  tests 

4.  ARC  or  "tab"  tests 

5.  pencil-paper  and  oral- interv iew  tests. 

The  major  difference  between  the  troubleshooting  research 
of  the  fifties  and  the  troubleshooting  research  of  the 
seventies  is  the  use  of  computers  for  data  gathering  in 
the  later  efforts. 
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Fink,  C.D.;  Shriver,  E.L.  (July  1978)  "Simulators  for 
Maintenance  Training."  Air  Force  Human  Resources  Laboratory, 
AFHRL-TR-78-27. 

This  report  reviews  past  and  present  applications  of 
simulation  to  maintenance  training.  Emphasis  is  on 
describing  issues,  problems  and  areas  for  future  research 
as  identified  by  recent  authors.  There  is  little 
evidence  from  the  literature  regarding  the  degree  to 
which  simulators  promote  trarisfer  of  training  to  the 
field.  Thus  far,  transfer  research  has  demonstrated  that 
simulators  can  increase  proficiency  during  and  at  the  end 
of  resident  training.  A  variety  of  research  has  also 
demonstrated  that  major  variations  in  resident  training 
cannot  be  correlated  with  job  proficiency  after  a  year  or 
more  in  the  field.  It  still  remains  to  be  shown 
conclusively  that  transfer  of  training  affects 
performance  during  the  first  weeks  and  months  on  the  job. 

If  transfer  does  occur,  it  should  have  its  most  important 
and  noticeable  impact  during  these  weeks.  There  is  a 
need,  therefore,  for  studies  which  examine  the 
relationship  between  resident  training,  especially  that 
supported  by  simulators,  and  on  the  job  proficiency 
during  the  first  one  to  three  months  on  the  job. 
Allocating  the  position  of  simulators  and  actual 
equipment  systems  in  maintenance  training  requires 
research  investigating  the  optimal  sequence  of  tasks  and 
training  aids  for  maintenance  training. 
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Foley,  J.P.,  Jr.  (1977)  "Overview  of  Advanced  Systems  Division 

Criterion  Research  (Maintenance)."  Air  Force  Human  Resources 

Laboratory,  AFHRL-TR-77-77. 

A  prime  but  seldom  considered  cause  of  the  current  high 
maintenance  cost  of  DOD  hardware,  and  thus  the  high 
ownership  cost  of  systems,  is  the  current  criteria  used 
by  personnel  systems  to  select,  train,  assign,  and 
promote  maintenance  personnel.  The  current  criteria 
emphasize  the  ability  to  obtain  high  scores  on 
paper-and-pencil  theory  and  job  knowledge  tests.  This 
paper  summarizes  the  many  studies  which  indicate  that 
such  tests  have  little  demonstrated  relationship  to 
ability  to  perform  job  tasks.  Table  1  summarizes  the 
current  research  on  performance  measurement  and  indicates 
for  each  citation  the  low  correlations  obtained  in  those 
experiments  for  various  maintenance  tasks  and 
paper-and-pencil  theory  tests,  job  knowledge  tests,  and 
school  grades.  Based  on  an  experiment  conducted  in  the 
past  by  AFHRL,  it  was  shown  that  graphic  symbolic  tests 
were  more  promising  than  symbolic  tests  in  measuring  job 
task  performance.  A  maintenance  task  taxonomy  is 
discussed  and  a  hierarchy  of  dependencies  is  developed 
for  electronic  maintenance  tasks.  This  conceptualization 
is  considered  necessary  before  identifying  the  relevant 
criteria  in  job  task  performance  tests. 
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King,  W.  (1978)  "New  Concepts  in  Maintenance  Training."  Aviation 
Engineering  and  Maintenance ,  2  (6):  24-26.  ‘  '  ~ 

This  article  discusses  the  present  status  of  Navy 
maintenance  training  in  light  of  recent  developments  in 
CAI  (Computer  Assisted  Instruction).  The  advantages  of 
2-D  computer  simulators  and  3-D  real  equipment  simulators 
for  troubleshooting  is  the  main  focus  of  example.  The 
author  presents  monetary  comparisons  between  actual 
equipment  trainers  (AET)  and  simulators  of  both  the 
computer  and  real  equipment  type.  The  best  example  is 
the  cost  comparison  for  the  training  of  radio  room 
technicians  for  the  Trident  submarine.  The  AET  costs  20 
million  dollars  compared  to  a  2  million  dollar  computer 
simulator.  King  believes  that  the  utilization  of 
computer  simulations  for  maintenance  training  has  the 
potential  to  surpass  the  savings  offered  by  simulators 
for  flight  training. 
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Standlee,  L.S.;  Popham,  W.J.;  Fattu,  N.A.  (1956)  A  Review  of 
Troubleshooting  Research .  Indiana  University:  institute  o 7 
Educational  Research. 

This  report  is  an  annotated  bibliography  of  the 
troubleshooting  literature  of  the  fifties.  It  contains  a 
brief  description  of  the  work  of  nearly  one  hundred 
authors.  The  report  is  indexed  with  the  following  terms: 
general  discussion,  proficiency  assessment  techniques, 
training  equipment  or  materials,  analysis  of  job 
activities,  evaluation  of  maintenance  personnel, 
experimentation,  bibliography.  This  report  is  invaluable 
to  persons  interested  in  the  early  efforts  of  fault 
diagnosis  research. 
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MANUALS  OF  ACTUAL  FAULT  DIAGNOSIS  OR  TRAINING  IN  FAULT  DIAGNOSIS 


AVCO  Lycoming.  (1975)  MAVCO  Lycoming  Reciprocating  Engine 
Troubleshooting  Guide."  Williamsport,  PA:  SSP-475. 

A  manufacturer  published  list  of  problems,  causes,  and 
solutions  that  represents  the  more  common  and  recurring 
problems  encountered  by  mechanics  when  troubleshooting. 

The  list  is  general  and  refers  to  turbocharged  and 
normally  aspirated  engines.  This  manual  is  written  for 
the  professional  mechanic  and  assumes  that  shop  manuals 
and  proper  tools  are  available. 
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Bean,  R.  (1974)  Petersen's  Basic  Automotive  Troubleshooting . 
Los  Angeles, CA:  Petersen  Publishing  Co  . 

This  manual  is  a  very  basic  guide  for  the  novice  or 
do-it-yourself  automotive  enthusiast.  It  includes  an 
easy  to  follow  symptom-cause  format  with  illustrations. 

The  book  is  divided  into  such  chapters  as  the  following: 
Troubleshooting  the  Brakes;  Troubleshooting  the  Ignition 
System;  Troublshooting  Automatic  Transmissions;  Engine 
Oil  Pressure  Problems;  and  Troubleshooting  with  a  Vacuum 
or  Compression  Gauge.  Simple  logic  trees  are  included 
for  the  starting,  ignition  and  fuel  systems. 
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Chilton  Book  Co.  (1972) 
Repair  4  Tune-Up  Guide. 
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"Troubleshooting."  Chilton  *  s  Mustang 
Radnor,  PA:  Chilton  Book  Company; 


A  section  of  the  manual  is  designed  to  aid  in  the  rapid 
diagnosis  of  engine  problems.  The  systematic  format  is 
used  to  diagnose  problems  ranging  from  engine  starting 
difficulties  to  the  need  for  engine  overhaul.  It  is 
assumed  that  the  user  is  equipped  with  basic  hand  tools 
and  test  equipment  (tach-dwell  meter,  timing  light, 
voltmeter,  and  ohmmeter).  Troubleshooting  is  divided 
into  two  sections.  The  first,  General  Diagnosis ,  is  used 
to  locate  the  problem  area.  Tn  the  second,  Specific 
Diagnosis,  the  problem  is  systematically  evaluated.  The 
format  presents  a  malfunction  system  followed  by  a  list 
of  tests  and  procedures.  The  proper  results  of  the  test 
are  provided  in  an  effort  to  aid  the  troubleshooter . 

15  pages 


Glenn,  H.T.  (1971)  Glenn*  s  Auto  Troubleshooting  Guide. 

Philadelphia,  PA:  Chilton  "Book  Co. 

This  manual  is  another  example  of  a  popular  book  written 
for  the  automotive  enthusiast  rather  than  the  experienced 
mechanic.  The  specific  areas  that  it  covers  are: 
emergencies,  hard  starting,  engine  failure,  excessive  oil 
consumption,  engine  noises,  overheating,  fuel  system 
trouble,  electrical  system  failure,  ignition  system 
failure,  drive,  transmission,  and  rear  axle  trouble,  and 
power  brake  and  power  steering  failure. 
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Howard  W.  Sams  4  Co.  (1973)  Tune-up  Service  with  Testing , 
Diagnosis  and  Troubleshooting  Indianapolis,  IN:  Howard  W.  Sams 
4  Co.,  Inc.  2nd  Edition. 

This  manual  covers  the  latest  emission  control  systems, 
electronic  ignition  systems,  electronic  test  equipment, 
and  conventional  engine  analysis  equipment.  The 
troubleshooting  section  presents  the  familiar 
symptom/cause  format. 
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Mitchell  Manuals,  Inc. 
Mitchell  Manuals,  Inc. 


(1977)  Diagnostic  Guide.  San  Diego,  CA 


The  book  contains  information  designed  to  aid  the 
professional  mechanic  in  diagnosis  and  repair  of  unusual 
automotive  problems.  Since  many  malfunctions  are  caused 
by  incorrect,  inoperative,  defective  or  missing 
components,  attention  is  focused  on  these  problems  and 
conditions.  All  information  presented  has  been  carefully 
compiled,  primarily  from  factory  service  bulletins, 
pinpointing  the  cause  of  specific  malfunctions.  This 
diagnostic  guide  is  designed  to  help  the  mechanic  avoid 
potential  problems  and  save  time. 

The  book  is  arranged  for  quick  reference  in  finding 
particular  information.  The  book's  main  divisions  are 
done  by  vehicle  manuf acturers :  American  Motors,  Buick, 
Cadillac,  etc.  Within  each  of  the  divisions  there  are 
three  areas  of  information: 

1 )  troubleshooting  and  diagnosis 

2)  jacking,  hoisting  and  towing 

3)  hood  latch  and  safety  release  locations 
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General  Motors  Corporation.  (1978)  Diagnosis  and  Repair  Manual 
Easy  to  Follow  Graphic  Troubleshooting  Procedures.  Southfield 
MI:  General  Motors  Corporation. 

This  manual  contains  graphic  mode  diagnosis  and  repair 
charts.  They  are  a  relatively  new  approach  to 
systematic  diagnosis  and  repair.  The  charts  are  easy  to 
use  since  they  require  minimal  reading.  The  sketches  of 
exact  engine  parts  allows  the  mechanic  to  proceed  to 
work  without  reading  a  manual,  although  on  complex 
repairs  the  book  emphasizes  that  a  manual  should  be 
used . 

These  diagnosis  procedures  apply  to  various  automotive 
systems  in  most  1970  thru  current  GM  cars  and  light-duty 
trucks-,  with  some  procedures  applying  to  even  earlier 
model  years.  The  procedures  are  based  on  the  fact  that 
the  vehicle  worked  right  at  one  time  and  the  problem  is 
due  to  time,  wear,  dirt  or  other  causes. 
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MATERIALS  INDIRECTLY  RELATED  TO  FAULT  DIAGNOSIS 


Berner,  W.S.;  Hamilton,  L.A.;  Best, 
Approach  to  Evaluating  Problem-Solving 
Journal  of  Medical  Education.  49:666-672. 


W.R.  (1974)  "A  New 
in  Medical  Students." 


This  paper  describes  the  development  of  a  new  technique 
to  assess  clinical-  problem-solving  skills  in  medical 
students.  The  procedure  enables  each  step  in  the 
solution  of  clinical  problems  to  be  evaluated 
independently  by  providing  feedback  to  students  at  each 
decision  point.  There  are  several  unique  features, 
including  the  incorporation  of  some  aspects  of  the 
problem-oriented  medical  record.  Students  were  required 
to  read  an  extensive  data  base  and  construct  problem 
lists,  order  diagnostic  tests,  and  plan  for  the 
management  of  the  patient.  The  technique  was  designed  to 
(a)  be  administered  to  large  groups  of  students,  (b)  be 
easily  scorable,  and  (c)  overcome  some  of  the 
difficulties  inherent  in  previously  developed  means  oi 
assessment . 
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Brown,  J.S.;  Burton,  R.R.  (1978)  "Diagnostic  Models  for 

Procedural  Bugs  in  Basic  Mathematical  Skills.”  Cognitive  Science. 

2(2) : 155-192.  - 

A  new  diagnostic  modeling  system  for  automatically 
synthesizing  a  deep-structure  model  of  a  student's 
misconceptions  of  bugs  in  his  basic  mathematical  skills 
provides  a  mechanism  for  explaining  why  a  student  is 
making  a  mistake  as  opposed  to  simply  identifying  the 
mistake.  This  report  is  divided  in  four  sections:  The 
first  provides  examples  of  the  problems  that  must  be 
handled  by  a  diagnostic  model.  It  then  introduces 
procedural  networks  as  a  general  framework  for 
representing  the  knowledge  underlying  a  skill.  The 
challenge  in  designing  this  representation  is  to  find  one 
that  facilitates  the  discovery  of  misconceptions  of  bugs 
existing  in  a  particular  student's  encoding  of  this 
knowledge.  The  second  section  discusses  some  of  the 
pedagogical  issues  that  have  emerged  from  the  use  of 
diagnostic  models  within  an  instructional  system.  This 
discussion  is  framed  in  the  context  of  a  computer-based 
tutoring/gaming  system  developed  to  teach  students  and 
student  teachers  how  to  diagnose  bugs  strategically  as 
well  as  how  to  provide  a  better  understanding  of  the 
underlying  structure  of  arithmetic  skills.  The  third 
section  describes  our  uses  of  an  executable  network  as  a 
tool  for  automatically  diagnosing  student  behavior,  for 
automatically  generating  "diagnostic"  tests,  and  for 
judging  the  diagnostic  quality  of  a  given  exam.  Icluded 
in  this  section  is  a  discussion  of  the  success  of  this 
system  in  diagnosing  1300  school  students  from  a  data 
base  of  20,000  test  items.  The  last  section  discusses 
future  research  directions. 
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Ellery,  G.L.  (1977)  "The  EE’s  Role  in  the  Garage  Diagnosis  of 
Vehicle  Faults."  Proceedings  of  1978  IEEE  Conference  on  Vehicular 
Technology,  Denver. 

The  complexity  of  modern  automobiles,  trucks  and  other 
internal  combustion  engines  demands  more  sophisticated, 
computerized  diagnostic  equipment.  The  requirement  of 
high  fuel  economy  and  low  hydrocarbon  emissions  further 
highlights  this  need.  The  author  -discusses  the  fact  that 
automobile  manufacturers  are  increasing  their  involvement 
in  the  design  and  construction  of  automated  test 
equipment.  The  R  &  D  for  such  equipment  is  the  combined 
effort  of  the  EE,  ME,  and  Automotive  Service  Engineer. 

Computer  controlled  test  equipment  which  relies  on 
micro-processor  technology  is  already  found  in  many 
automotive  repair  facilities,  but  the  sophistication  of 
such  equipment  is  rapidly  increasing.  The  author  offers 
examples  of  electrical  analysis  of  starter-motor-current 
oscillation  and  analysis  of  ignition  spark  amplitude  to 
determine  engine  compression. 

3  pages 


McGuire,  C.H. ;  Babbott,  D.  (1967)  "Simulation  Technique  in  the 
Measurement  of  Problem-Solving  Skills."  Journal  of  Educational 
Measurement .  4:1-10. 

Employing  simulation  technique  analogous  to  that  utilized 
in  management  games  and  military  exercises,  the  authors 
have  directed  the  development  of  a  series  of  branched 
problems  in  patient  management  requiring  sequential 
analysis  and  decision.  These  problems  are  designed  to 
measure  aspects  of  behavior  'defined  by  a  criterion  group 
as  essential  components  of  clinical  competence.  The 
materials  are  suitable  to  either  paper  and  pencil  or 
on-line  application.  The  validity  and  reliability  of 
simulations  is  also  discussed. 
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McGuire,  C.H.;  Solomon,  L.M.;  Bashook,  P.G.  (1976). 
Construction  and  (Jse  of  Written  Simulation .  United  States  of 
America :  TEe  Psychological  Corporation  . 

This  work  is  a  comprehensive  discussion  of  written 
simulations.  The  discussion  ranges  from  basic 
definitions  through  construction,  use  and  evaluation  of 
simulations.  The  book  is  additionally  applicable  to 
computer  simulations,  or  real  equipment  simulations. 
Sample  problems  are  presented  throughout  the  book. 


Murray,  T.S.;  Cupples,  R.W.;  Barber,  J.H. ;  Dunn,  W.R.;  Scott, 
D.B.;  Hannay,  D.R.  (1977)  "Teaching  Decision  Making  to  Medical 
Undergraduates  by  Computer-Assisted  Learning."  Medical  Education. 
11:262-264.  - 

This  paper  describes  a  computer-assisted  learning  (CAL) 
program  that  is  used  on  an  open-acess  basis  by  students 
at  the  University  of  Glasgow  medical  school.  The  authors 
report  an  experiment  in  which  the  experimental  group 
(ns12)  was  presented  with  on-line  case  studies  in  which 
they  had  to  decide  on  patient  treatment.  Upon  completion 
of  the  experiment,  the  experimental  group  scored  higher 
on  a  factual  knowledge  test  and  had  improved  decision 
making  skills  based  on  comparison  of  their  scores  to 
those  of  medical  experts.  The  subjects  felt  that  CAL 
enabled  them  to  consider  and  discuss  clinical  decisions 
in  a  way  that  is  both  more  realistic  and  more  instructive 
than  is  possible  in  the  more  orthodox  clinical  teaching. 
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Swain,  A.D.  (1961)  Factors  Affecting  Degree  of  Automation  in 
Test  and  Checkout  Equipment.  Stamford ,  Conn.:  £>unlap  anU 
Associates ,  Inc . 

This  report  deals  primarily  with  the  human  factors 
problems  attendant  to  the  increasing  automation  of  weapon 
system  test  and  checkout  actions.  The  purpose  of  the 
report  is  to  identify  unfavorable  tactical  situations 
resulting  from  less-than-optimum  allocation  of  test  and 
checkout  actions,  from  inadequate  design  of  automatic 
checkout  equipment  (ACE),  or  from  particular  management 
or  other  practices  related  to  the  utilization  of  ACE. 
These  cases  suggest  ways  of  avoiding  such  situations  in 
the  future.  While  most  of  the  examples  are  taken  from 
Air  Force  weapon  systems,  some  are  taken  from  Army  and 
Navy  weapon  systems  and  equipment;  and  the  principles 
and  guidelines  are  generally  applicable  to  all  the 
services.  It  is  only  the  application  which  should  be 
different,  as  considerable  differences  often  occur 
between  services  in  such  areas  as  training,  manpower 
selection,  environmental  conditions,  and  logistics 
support  (to  name  only  a  few  differences  in  system 
criteria)  . 
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Wollmer,  R.D.;  Bond,  N.A.,  Jr.  (1975)  "Evaluation  of  a 
Markov-Decision  Model  for  Instructional  Sequence  Optimization." 
Behavioral  Technology  Laborator ies ,  Report  No.  76,  University  of 
Southern  California,  Los  Angeles,  CA. 

Two  CAI  programs  in  electronics  and  trigonometry  were 
written  to  test  the  Wollmer  Markov  Model  for  optimizing 
hierarchical  learning;  calibration  samples  totalling  110 
students  completed  these  programs.  Since  the  model 
postulated  that  transfer  effects  would  be  a  function  of 
amount  of  practice,  half  the  students  were  required  to 
complete  one  practice  problem  successfully  before  moving 
to  the  next  stage;  the  other  half  had  to  do  two  practice 
problems  successfully. 

All  students  completed  the  courses  satisfactorily. 
Practice  effects  were  small;  students  who  had  one 
success  in  each  stage  did  about  as  well  as  those  who  had 
two  successes.  The  Wollmer  model  was  thus  not  suitable 
for  optimizing  instruction,  in  terms  of  minimizing 
overall  time,  in  these  particular  courses.  Perhaps  the 
main  reason  for  this  result  was  that,  as  the  student 
works  up  to  the  top  of  the  hierarchy,  the  sheer  number  of 
subskills  involved  in  the  final  task  becomes  a  major 
determinant  of  performance  time,  and  number  of  practice 
trials  has  a  relatively  minor  effect,  unless  a  very  large 
number  of  practice  trials  is  given.  To  see  if  a  much 
larger  amount  of  practice  would  affect  time  to  perform, 
three  additional  subjects  were  run  with  the  electronics 
problems  under  the  condition  that  5  correct  solutions 
were  required  at  each  of  11  levels.  It  was  found  that  a 
much  larger  training  time  was  required  to  get  a  2-3 
minute  improvement  at  the  final  level.  Because  of  this 
relative  inefficiency,  the  Wollmer  model  would  not 
prescribe  such  extensive  additional  practice. 
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